深度加强学习（RL）代理在一系列复杂的控制任务中变得越来越精通。然而，由于引入黑盒功能，代理的行为通常很难解释，使得难以获得用户的信任。虽然存在一些基于视觉的RL的有趣的解释方法，但大多数人都无法发现时间因果信息，提高其可靠性的问题。为了解决这个问题，我们提出了一个时间空间因果解释（TSCI）模型，以了解代理人的长期行为，这对于连续决策至关重要。 TSCI模型构建了颞会因果关系的制定，这反映了连续观测结果与RL代理的决策之间的时间因果关系。然后，采用单独的因果发现网络来识别时间空间因果特征，这被限制为满足时间因果关系。 TSCI模型适用于复发代理，可用于发现培训效率高效率的因果特征。经验结果表明，TSCI模型可以产生高分辨率和敏锐的关注掩模，以突出大多数关于视觉的RL代理如何顺序决策的最大证据的任务相关的时间空间信息。此外，我们还表明，我们的方法能够为从时刻视角提供有价值的基于视觉的RL代理的因果解释。

在自我监督对比度学习范式下，小型模型表现得很差。现有方法通常采用大型现成模型，通过蒸馏将知识转移到小型。尽管有效率，但由于部署大型模型的巨大计算费用，蒸馏基方法可能不适合某些资源限制方案。在本文中，我们研究了没有蒸馏信号的自我监督小型模型的问题。我们首先评估小型模型的代表空间，并进行两个不可忽略的观察：（i）小型型号可以完成借口任务，而无需过度拟合，尽管它们有限，并且（ii）他们普遍遭受聚类问题的问题。然后我们验证了多个被认为减轻过分聚类现象的假设。最后，我们结合了验证的技术，提高了五种小型架构的基线性能，具有相当大的边缘，这表明即使没有蒸馏信号，培训小自我监督的对比模型也是可行的。该代码可在\ texit {https://github.com/wodeice/sl-small}中获得。

无模型的深度加强学习（RL）算法已广泛用于一系列复杂的控制任务。然而，慢的收敛和样本效率低下在R1中仍然具有挑战性，特别是在处理连续和高维状态空间时。为了解决这个问题，我们提出了一种通过绘制潜在的Anderson加速度（RAA）的想法，提出了一种无模型的非政策深度RL算法的一般加速方法，这是加速扰动解决固定点问题的有效方法。具体来说，我们首先解释如何使用Anderson加速直接应用策略迭代。然后，我们通过引入正则化术语来扩展RAA，以控制函数近似误差引起的扰动的影响。我们进一步提出了两种策略，即逐步更新和自适应重启，以提高性能。我们的方法的有效性在各种基准任务中评估，包括Atari 2600和Mujoco。实验结果表明，我们的方法大大提高了最先进的深度RL算法的学习速度和最终性能。

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the try-on person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-of-the-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.

Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q-learning algorithm, which integrates principal component analysis (PCA) with deep Q-learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.

As natural language processing (NLP) for gender bias becomes a significant interdisciplinary topic, the prevalent data-driven techniques such as large-scale language models suffer from data inadequacy and biased corpus, especially for languages with insufficient resources such as Chinese. To this end, we propose a Chinese cOrpus foR Gender bIas Probing and Mitigation CORGI-PM, which contains 32.9k sentences with high-quality labels derived by following an annotation scheme specifically developed for gender bias in the Chinese context. Moreover, we address three challenges for automatic textual gender bias mitigation, which requires the models to detect, classify, and mitigate textual gender bias. We also conduct experiments with state-of-the-art language models to provide baselines. To our best knowledge, CORGI-PM is the first sentence-level Chinese corpus for gender bias probing and mitigation.

Off-policy evaluation (OPE) is a method for estimating the return of a target policy using some pre-collected observational data generated by a potentially different behavior policy. In some cases, there may be unmeasured variables that can confound the action-reward or action-next-state relationships, rendering many existing OPE approaches ineffective. This paper develops an instrumental variable (IV)-based method for consistent OPE in confounded Markov decision processes (MDPs). Similar to single-stage decision making, we show that IV enables us to correctly identify the target policy's value in infinite horizon settings as well. Furthermore, we propose an efficient and robust value estimator and illustrate its effectiveness through extensive simulations and analysis of real data from a world-leading short-video platform.

Off-Policy evaluation (OPE) is concerned with evaluating a new target policy using offline data generated by a potentially different behavior policy. It is critical in a number of sequential decision making problems ranging from healthcare to technology industries. Most of the work in existing literature is focused on evaluating the mean outcome of a given policy, and ignores the variability of the outcome. However, in a variety of applications, criteria other than the mean may be more sensible. For example, when the reward distribution is skewed and asymmetric, quantile-based metrics are often preferred for their robustness. In this paper, we propose a doubly-robust inference procedure for quantile OPE in sequential decision making and study its asymptotic properties. In particular, we propose utilizing state-of-the-art deep conditional generative learning methods to handle parameter-dependent nuisance function estimation. We demonstrate the advantages of this proposed estimator through both simulations and a real-world dataset from a short-video platform. In particular, we find that our proposed estimator outperforms classical OPE estimators for the mean in settings with heavy-tailed reward distributions.

The ability to jointly learn from multiple modalities, such as text, audio, and visual data, is a defining feature of intelligent systems. While there have been promising advances in designing neural networks to harness multimodal data, the enormous success of data augmentation currently remains limited to single-modality tasks like image classification. Indeed, it is particularly difficult to augment each modality while preserving the overall semantic structure of the data; for example, a caption may no longer be a good description of an image after standard augmentations have been applied, such as translation. Moreover, it is challenging to specify reasonable transformations that are not tailored to a particular modality. In this paper, we introduce LeMDA, Learning Multimodal Data Augmentation, an easy-to-use method that automatically learns to jointly augment multimodal data in feature space, with no constraints on the identities of the modalities or the relationship between modalities. We show that LeMDA can (1) profoundly improve the performance of multimodal deep learning architectures, (2) apply to combinations of modalities that have not been previously considered, and (3) achieve state-of-the-art results on a wide range of applications comprised of image, text, and tabular data.